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THE PURPOSE OF THE RESEARCH WAS TO OBSERVE THE 
CONSISTENCY OF INTRA-INDIVIDUAL VARIABILITY FROM ONE TASK TO 
ANOTHER, TO DETERMINE THE RELIABILITY OF OBSERVATIONS OF 
INTRA-INDIVIDUAL VARIABILITY, AND TO OBSERVE THE EXTENT TO 
WHICH THE ACCURACY OF PREDICTION VARIED WITH THE CONSISTENCY 
OF THE INDIVIDUAL'S BEHAVIOR. MALE ENGINEERING STUDENTS AT 
THE UNIVERSITY OF MINNESOTA WERE ADMINISTERED A BATTERY OF 
SIX BRIEF, INDEPENDENT TESTS ON EACH OF TWENTY DAYS. MOST OF 
THE OBSERVATIONS OF INTRA-INDIVIDUAL VARIABILITY WERE 
RELIABLE. VARIABILITY OVER TIME ON SOME TASKS IS RELATED TO 
VARIABILITY ON OTHER TASKS, BUT THESE RELATIONSHIPS ARE ONLY 
MODERATE AND THE HIGHEST CORRELATIONS BETWEEN ANY TWO OF THE 
VARIANCE INDICES WAS ONLY ABOUT .50. INTRA-INDIVIDUAL 
VARIABILITY IS NOT SPECIFIC TO EACH TASK BUT NEITHER IS THERE 
A BROADLY GENERALIZED CHARACTERISTIC OF SUCH VARIABILITY. A 
RELATIONSHIP WAS OBSERVED BETWEEN THE VARIABILITY OVER TIME 
OF A PERSON'S BEHAVIOR ON SIMPLE TASKS AND THE EFFECTIVENESS 
WITH WHICH HIS ACADEMIC PERFORMANCE CAN BE PREDICTED. SOME OF 
THE ANALYSES REVEALED ERRORS OF PREDICTION SIGNIFICANTLY 
SMALLER FOR THE LESS VARIABLE GROUP. CONSIDERED IN THE LIGHT 
OF RESULTS OF OTHER RESEARCH, THESE RESULTS SUGGEST THAT A 
COMPLEX BUT PRESUMEDLY USEFUL RELATIONSHIP EXITS BETWEEN THE 
CONSISTENCY OF AN INDIVIDUAL'S BEHAVIOR AND THE EXTENT TO 
WHICH HIS ACADEMIC BEHAVIOR IS PREDICTABLE. (AUTHOR) 
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^purpose of the research was to observe the consistency of intra-individual 
variability from one task to another, to determine the reliability of observations 
of intra-individual variability, and to observe the extent to which the accuracy 
of prediction varied with the consistency of the individual’s behavior. One hundreli 
male engineering students at the University of Minnesota were administered a batter jr 
of six brief independent tests, the Repeated Psychological Measurements, on each of 
twenty days, a different form of, each test being used each day. For each subject 
variance indiced were calculated on each of the six tests and on the total of seven 
tests and the relationships between variance indices were observed, the reliabili- 
ties of the indices determined, and the relationships between intra-individual 
variability and the prediction of academic success observed. 

Most of the observations of intra-individiaal variability were reliable. Varia- 
bility over timeon some tasks is related to variability on other tasks, but these 
relationships are only moderate and the highest correlations between any two of the 
variance indices was only about ,50* Intra— individual variability is not specific 
to each task but neither is there a broadly generalized characteristic of such 
variability, A relationship was observed between the variability over time of a 
persons ^®havior on simple tasks and the effectiveness with which his academic per- 
foi?mance can be predicted. Some of the analyses revealed errors of prediction sig- 
nificantly smaller for the less variable group. One analysis showed that the cor- 
relation between predicted grades and obtained grades was ,67 for a less variable 
gro^p, as compared .to ,l6,.for # more; vart®^l® Considered in the light of 

SoQul-hfl ftf rAAiilta thVt a hnmplaT blit pmaaMm a i^lv u aa 



ful relationship exists between the consistency of an individual’s behavior andi 
he extent to ^9 p re cii.Qt 4 h 3 ,cti. 



INTRA-INDIVIDUAL VARIABILITY OF BEHAVIOR AND THE] 
PREDICTABILITY OF ACiJDEI-IIC SUCCESS 



Project No. 6-8694 
Contract No. OSC -3-7-068694-2082 



Ralph P, Berdie 



February I 968 



The research reported herein v;as perforuied pursuant to a 
contract with the Office of Education, U. 3. Department 
of Health, Educat5.on, and Welfare. Contractors under- 
taking such projects under Government sponsorship are 
encouraged to express freely their professional judgment 
in the conduct of the project. Points of view or opinions 
stated do not, therefore, necessarily represent official 
Office 0 ; Education position or policy. 



University of Minnesota 



Minneapolis, Mnnesota 



The Oonyit'tency and Genoralizability of 



Intra-individual Variability^'- 



bv 



Ralph F. Berdie 
Office of the Dean of Students 



University of Minneso'fca 



The extent to v/hich an individual’s behavior is consistently var- 
iable on a given task or on different tasks may provide cues as to the 
effectiveness of his behavior. The variability studied here correspoiids 
to that descr'ibed by Fiske (Fiske & Maddi, p.527)j ”... one form of 
variability, the variation in the behavior of a given organism at differ- 
ent times but under the same external conditions.” This form of varia- 
bility is to be contrasted v.dth that disc'assed by Hull ( 1927 ) v;ho v/as 
concerned v/ith variability in the amo-unt of different traits possessed 
by an individual, and also to be contrasted v/ith the variability dis- 
cussed by V/echsler (l950) v;ho './as concerned v/ith the variability of a 
given trait v/ithin the population. 

Theore’bically, an individual cannot reproduce a behavior iden-bically 
insofar as once he has performed a task, repetition of that task must be 
influenced by its prior performance. In spite of the impossibility of 
s'ciadying the individual's variabili'by v.diile performing the same task, 
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his variahillijr can he studied while pe?:f orming a homogenous group of 
tasks. Tasks and situations change from performance to performance, hut 
these can he ordered into highly similar categories and variability of 
behavior so studied. 

Piske provided one theoretical basis for the analysis of variability. 

He regarded individual variability as having a coping function vMch effects 
the organism’s adaptability. V/hen several alternate behaviors are avail- 
able to the organism, its inherent variability increases the likelihood 
that the organism eventually v;ill select and adopt .the response which 
best copes v;ith a given situati.on. 



I'iske (Picke Maddi, I961) asked several relevant questions con- 
cerning variability: 

1 . What is the amount of such variability around the central 
tendency? 

2 . Hoes the amount of variability vary with the value of the 
central tendency? 

5. Hoes the extent of variability differ for different tests (tasks)? 

\ 

4. Ho individual subjects show different amounts of variability? 

5. V/ith what are these individiuil differences in variability 
associated? 

In an earlier study, the author (Berdie, I961) found that variability 
on a task involving advanced high school mathcMiiaticj ax^peared consistent 
within the individual and that this variability might be related to the 
extent to which a person's college achievement could be predicted on the 
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of aptitrao tos-by. Confjocjuently the present research found its 
immediate origin in the last of the above listed questions but eventually 
ecjually involved questions concerning tb.e consistency ano. {^enerali/iabill cy 
of intra-individual variability . 

The two ques bions approached here v/ere 1. Can the variability 
over time of an individual’s behavior be reliably and consistently ob- 
served? 2. To what extent are persons variable over time on one task 
also variable olf other tasks? 

Once these two questions are ansvjered, and if variability over tine 
can be reliably observed and if variability over time is not specific 
to each task, then the variability of an individual may be a useful con- 
cept in understanding his behavior. For ox^?.^}ple, variable persons may 
be Vfiore or less pre0.ictable, may be more or less task oriented, or may be 
described by other personality chara,cteristics . 



Method 



Tasks . 

The behavior observed here consisted of responses to six of the 
Repetitnve Psychometric Measures (scores on these tests) developed by 
Moran and Mefferd (l959). These authors reported that each of the six 
tests represented a distinct factor derived from repeated factor analytic 
studies. The Aiming test (a) consists of fifteen rov/s each containing 
t'wenty circles and the circles are in ro'.s connected sequentially by b , 
line. The subject places the test on a piece of corregated paper and the 
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taak io t-o punch a hole inside as many circles as possible v/ithin ninety 
seconds without touching the circles. The subjects in this experiment 
used a styliis consisting of a pencil sized piece of wood with a thin pin 
point at one end. The tost involves the ability to carry out quickly 
and precisely a series of movements depending on eye-hand coordination 
and requires careful placement of the stylus point in specified locations. 
This particular test is not mentioned in Prench’s report v/hich does 

refer to each of the other five, tests. In a letter from Moran to the author 
the Aiming test v/as described as one of several dexterity tests that did 
not correlate with each other. In their original paper, Moran and Mefferd 
(1959) quoted French as saying, "The ex.tent to v;hich this specific factor 
(Aiming) can be generalized to other abilities is unknovm," Moran commented 
that the same form of this test given repeatedly v/ould serve the purpose 
of a repetitive measure. 

The Flexibility of Closure test (FC) requires the subject to copy 
thirty-six geometric figures into matrices of dots. Each test form con- 
tains thirty-six figures and the subject is allov/ed three minutes. The 
task, as described by the authors, is to retain the image of a specified 
configuration despite the influence of other distracting configurations 
in the perceptual field. The Number Facility test (NF) is similar to 
French’s N factor and consists of ninety problems each requiring the 
addition of three two digit numbers. 

The Perceptual Speed te.st (PS) requires the identification of v/ell 
knovm symbols in a mass of material. This test is not described in French's 
1963 volume. The test consists of rows of thirty digits v/ith an encircled 
digit at the left of each row and the task is to cross out every digj.t in 
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Tho time -.imit siDecified by Moran 



the rov; sitnila}? to the encircled dicit. 
and Mefierd is tv;o and one half minutes but early experience v/ith this test 
using the subjects employed in this research suggested that too many siib- 
jects completed the test wrbh this time lij^at and the time limit was re- 
duced to one and one half minutes. 

The Speed of Closure test (SO) measures the ability to unify an appar- 
ently disparate perceptual field into a single percept and it v/as a marker 
test in French’s 19 b 4 kit but not included in the I963 kit. Bach form con- 
sists of tv/enty-tv/o lines and each line has letters in it apparently arranged 
at random but containing from tv/o to four four letter words which are to be 
encircled. The final test, Yisualisation (v), consists of tangled lines 
v-zhich must be followed visually from their start to their finish. French 
calls this the “choosing a path test” and lables the factor spatial 
scanning; he does not call it a visualization factor. 

For each of these tests Koran and Mefferd developed tventy different 
forms with the original intent that the forms would be equivalent. Later 
s-tu.dy, however, indicated that on each of the tests but Number Facility 
the alternate forms v;ere reliably different (Koran, Ivimble, & Kefferd, 

1964) and correction factors v/ere provided for the tvjenty alternate foriiis 
of these five tests . These correction factors v/ere not used in this study , 
in light of the experimental design. 

The test authors rej)ortGd test-retest reliabilities comparing scores 
on form one and form two, ranging from .72 to . 94 * Intercorrelations of 
the six tests, using only form one, ranged from .09 to .44. Considering 
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tho purposes for vMch these tests v/ero to be used here, they appeared 
to be adequately reliable and sufficiently independent from one another. 
Sample . 

The population from which the subjects \ioro drawn consisted of fresh- 
men entering the University of Minnesota Institute of Technology in the 
Pall of 1966. All freshmen were informed of the possibility of partici- 
pating in the experiment and from those who volunteered, subjects were 
selected on the basis of class schedules, availability of data, and prox- 
imity tD campus. Each subject v/as paid a total of forty dollars and sub- 
jects appeared v/ell motivated. The subjects tended to be somewhat superior 
academically to the total Pall quarter entering class because the experi- 
ment was conducted during the second quarter and involved only students 
who. survived the Pall quarter and reti^.rned. Por the most part the subjects 
consisted of fairly representative bright college students who had sur- 
vived at least one demanding aca-demic quarter and who v/ere motivated to 
earn forty dollars by participating in an experiment that would cause them 
no stress or discomfort. 

Procedures . 

The experiment v/as conducted in a- vrell isolated sub-basement roorn 
v/ith overhead lights and lamps arranged so that illu.mination v/as not 
brilliant but subjects could see comfortably. No noise from outside the 
building penetrated the room and little traffic passed in the corridor 
outside of the door. Temperature in the room v/as constant and comfort- 
able although v/hen the door v/as clovsed there v/as little ventilation. 
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Insofar as the snboects regained in tho room for periods of only twenty 
minutes and the door \‘is,o kept open for at least one half hour betvveen 
sessions, lack of ventilation produced no discomfort. 

Subjects V.. e seated in the center of the room in classroom chairs with 
arm tablets. They v/ere divi(3ed into five ^i,'roups and a {_,roup v/as besced 
each day at a.rn., 12i^0 noon, p.m., 2 ! 50 , and 5*30. Assi 2 ,ni:iento 

to time periods v/ere based on the class schedules submibted by subjects. 

Approximately one~fifth of the subjects took form one of bhe tes o on 
the first day, form two on the second, etc. Another group of subjects took 
form five on the first day, form six on the second day, and on the tv/enbiei;h 
day took form four. One group of subjects starbeci on form nine, anobhe.'. 
group on form thirteen, and another group on form seventeen, in order to 
provide some randomization of form-sequence influence. Within each time 
session, students v/e 3 ?e randomly assignee to sequence groiips. 

At the first session , the experimenter read to each group an intro- 
ductory statement and a trained and experienced psychomc brist bhen read one 
test instructions, adninisterei the practice exerciser provided by Horan 
and Mefferd, and administered the tests. 

Testing schedules for each group were ar3?anged Monday through Friday 
for four successive v/eeks a.nd subjects v/ho missed sessions made them up 
during the fifth week. Of the one hundred subjects, sixty- two attended 
daily ruid the romaining thirty-eight accounted for a total o;l seventy-ci-'. 
absoncGS latoj.’ made up. Twenty-tvjo of these thri?ty-eight persons \/cro ab- 
sent once, others v;ero absent from two to eignt times. 
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alvnijTG attend the sesf.:ion to v/hich they originally had "been assigned and 
sixty-three such time changes v/ere ohsor\red out of the tv/o thousand subject- 
session attendances. Over ninety-five percent of the tests v/ere c\d!ninistered 
to the subjects at the time of day originally scheduled. 

At the last session, at the completion of the last form of the last 
test, each subject completed a questionnaire reporting his reactions to the 
tasks and his perceptions of the purpose of the experiment. The subjects 
v/ere told at the first session that the purpose of the experiment v/as to 
compare the psychological characteristics, as measured by these tests, of 
students in technology and science to those of other students.' 

Analysis . 

The tv/elve thousand test papers v/ere scored by research assistants 
and v/hen scoring v/as completed, tv/o hundred papers for each test v/ere 
drav/n, representing all of the tv/enty forms, and these v/ere rescored. The 
original scores v/ere compared to those obtained by rescoring and frequencies 
of errors of various sizes v/ere tabulated and correlations determined. 
Scoring v/as judged to be adequate for five of those six tests but all t'./o 
thoivsand of the 'Number Facility tests v/ere res cored. The test scores v/ere 
then entered on basic record cards for each student, verified, and then 
punched and verified on IBM cards. 

For each of the six tests a variability index v/ns computed for each 
student. This consisted of the variance (8 'B') of the tv/enty raw scores 
derived from the tv/enty forms of the tests. At the same time, for each of 
the six tests a mean score v/ac computed for each student, this consis.ting 
of the mean of the tv/enty scores derived from the t'-'enty forms. 
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Then, in orde-'? to faiciXit'-ite comparisons hoti-.-een tests and to provide 
a basis for obtainin^^ a total variance index, each raw score was transiorned 
to a standard score, nsin^.; a mean equivalent to fifty and a standard de\la- 
tion equivalent to ten, based on the distribution of one hundred scores of 
each form of each test. For example, the one hundred scores on form one 
of the Aiming tost were selected, the luean and standard deviation c?i,lc'>'la.tGo., 
and for each student, his raw score on form one v/as transformed to a standard 
score based on this distribution. Then, for each of the six tests a var- 
iability index for each student v/as computed, alon" with a mean index, barsed 
on the tv/enty standard scores. A seventh variance and mean index were cal- 
culated for each student, based on all one hundred and tv/enty of the scores. 

Thus, for each student six variance indices and six mean indices based 
on ravj scores were available, and seven variance indices and seven mean in- 
dices were available based on standard scores. 

Th:'. consistency of the variability index v/as revealed by v/hat corresponds 
to an odd/even reliability coefficient. The scores on each of the ten odd 
numbered forms were used to provide a variance and a mean and the scorec; 
on each of the ten oven numbered forms provided comparable indices. On each 
test, each subject had two variance indices and two mean indices and in each 
instance, the correlation v/as calculated between the even numbered form and 
the odd numbered form, indices. The analysis v/as done first usin^^ rav/ scores 
and then standard scores. 

The correlation coefficients then v/oro calculated betv/oen the va,rianoo 
indices on the six tests, first usina; the indices based on raw scorofj and 
then the indices based on standard scores. The mean indices v/:-re anoly2;od 
similarly. 
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Relia’bili'by o.f.“ I'tidicefi. 

I'a’Dlo 1 shov/r: the corrolcvbiovis l:io'bv;ooii vai,’j.r‘ncc aiid niean indices based 
on odd .lumbered and oven nui.-iborod lormS} ■u.s.iii<^ rav’ sco/’es* iv..D.lo p/.oseiio».’ 
similcn? in.Corna’bion based on s'tandnrd scores ano, inclv.des .Lux o/. n;.;. G.Lon on 
'blie "bcbal score } vdiioli consisi'bs oi 'blio sum ol *bhc siandaro. sv./Cr(’jc> for uh.e 
si-x: tes-bs. Ibo test scores arc reliable, as shorn by the mean score correla- 
tions v.'hich ran^’G from .96 to .99* 

•Two of the variance indices, one based on Ainin- and tlio other on 
Number facility, shovr relatively hic^’h consistency; tvro, Percox/bual Speed 
and Speed of Closure, provide correlation coe.fficionts in the mid-fii ties . 
Visualisation xirovides the lov;est reliability coofficienb, »25* goc-.I 

variance index provides a correlation of .89, sw;£'Gstinc? that \/hatevor this 
is, it is an index that can be obtained rather consistently. 

These estimates of rcliabiliby arc based on ten scores. V;hen \ve use 
the Spearman-Brown proplnecy forniu].a, reliability estimates of the varia- 
bility index based on tv/enty scores are*. Aiming,', . 91 : Plex:Lbiln.ty of 
Closure, .! 36 : teiber Vecility, . 89 ; Perceptual Speed, , 71 ; Speed of Closure, 

• 75* and Visualisation, ./jO. One c-j-n concd.uae tha'c on A.im.Ln{y and buriiboL 
lb*,c',i lity va3?iability v.d'.thin persons tends to be rem-arkably consi^rbenb anvl 
consistency o.f va.idabi.Lity is founo. on all other cesos. 

The a-utlior (.IPrcdio, I96I), in a previous study, observed the relia- 
bility of a similar variance iiiefirAiro br.scc! on ton sul'i-sooros of a mathe- 
6iaVj.os t'.ohievo’ftc.ir'o ti.-ril;. 'i'he rol;i.ab;n.j.t'/ ct.c.ffi(;i<n!-fc:5 for v^riow.: f;roup.'.; 
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Incer'k Table 1 abcat here 



Insert Table 2 aboirb here 



in -bhat stady ran-ed about . 90 . Uain.:-; a sorr.cv/hat similar incthod of analyooG 
of varied repeated personality assessiiiont data, Pis):e (199?) reported odd/ 
even reliabilities extending from .46 to *3^, His results also suggosbed 
that the extent of consistency of a person’s variability depended in pare 
on the ts-sk or instrument used. 

Relationships betvjcon Variance Indices . 

Table 5 shov/s the inteDi'Correlatiions betvjeen the six variance inaices 
based on raw scores and Table 4 similar intercorrelations of the seven in- 
dices, including the total variance index, based on standard scores. In 
Table 3, of the fifteen correlations, four were significant beyond bhe .01 
level of probability, one between the .05 and .0:i level. The variances lor 
Aiming and Number Facility correlated .47? between Aiming and opeed of 
Closure, .28, betv/een Speed of Closure and Number Facility, .22, and be- 
tween Number Facility and Visu£-,lization, .27. The highest intercorrclatron 
was found bet\/eon the two variance indices having the highest reliabiliby 
and the intercorrelations must be examined in light of the reliaDil:i.tier> 
of the variance indJ.ces. 

Using the uncorrectod reliabilities based on the odd and oven nurnborod 
forms, and correobing the intor-test variance correlations for attenuation 
(unreliability) , the correlation between the variances for Ai.ming and 
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Ijuiiiljcj!’ ?*t).cilil"by iincx'GCACGS fi?o‘iA *^{1 *5^? l^ob\.’ooii A.L’iin.n2 <Anct. ijpccJ. ol* 

CloDurc from .28 to .41, 'bctvJCGn Hirn'bor Ij'acility and Speed of Closure from 
.22 to ,3i3, between HiUhbor Pacxlity and Yisualization from .27 to .60. 
•rheno oo-i?rel9.t;i.on5j suG’S^ot that some of the observed indcp'endonce betv/een 
tile v:-n?;>.ances is duo to the unreliabilities of the varii-,ncc indico:". 



Insert 0?able 5 about here 



Insert Table 4 about here 



The reliability coefficients themselves are rivlnimutn estimates and one 
can correct them a? v;e have done before by applyin.;;; the Spearman-I3rown 
prophecy formula. Unlimited corrections of this sort to statistical data 
lead to a morass of difficulty, particulo.rly v.'hen one is concerned v;ii-a 
prediction, but in this instance we are concerned v.dth arr;lving at soeio 
estimate as to relationships betv/oon variances , and these should bo bason 
on the best reliability estimates. Table 5 shows the intercorrelations 
of variance indices, vjhere the correlations are corrected for attenuation 
and the reliability coefficients used have been corrected vrith the Bpearnan- 
Urown prophecy formula. This table (ylves an optimal •estin),ate of the rela- 
tionships. 

Recof.nizin^,' the cpiostionable assumptions that have to oo made -./ikh 
those two corrections entcrino' into the coofficionts , the table reves'le 
that the variance on each of the tests is to some extent related bo uiio 
v<arianco on one or more of the othe?: tests. The variance index on i!u.Abo/. 
Facility is si^^nificantly cor’related with the index of each of thoobher 



Insert Table 5 £tbout here 



five tests. The varlcincoo on PlerJ.bility of Closure and Speed of Closure 
are related to variances on four of the other five tests. Perceptual Speed 
variance is correlated to three of the other indices, and tv/o of the coei'fi- 
dents are negative, and Aiding and Yisualiaation variances each are correla 
vdth tv/o of the other variance indices. The Aiming; and Number Indices have 
the highest reliabilities, are the most highly inter-correlated, and the 
Number Facility index is significantly related with each other index, althou 
the correlations are small. The best indication of the variance dotpain may 
be provided by the Number Facility and Aimdng tests. 



Discussion 



These resu.lts suggest that intra-individual variability is not specific 
to each task and neither is there a strongly generalised characteristic of 
variability that extends over a broad variety of tasks. Rather, the con- 
clusion is that the variability of a person on one task is somewhat related 
to his variability on certain other tasks and if one is to speak of such 
variability for a person, one must specify the tasks on v/hich statements 
are based. If more reliable means can be developed for observing in'br.a- 
individual variability, better defined clusters of tasks may ajjpear but at 
P'resent from (Umong the tasks observed here the tasks measured by the Aiming 
and Number Facility tests provide the best indicators of variabilityo 

The interpretation of these findings depends on other observations and 
analyses. A series of analyses of v.ariance revealed that the test forms 
are not ecjulveJ.ent and also that a significant practice or learning effect 



v/as prcfjent insofar as on all six of ihe -keo-bs daily moan scores for the 
group tended to increase from the hoG’inning to the end of the experiment. 

On five of the tests there was no evidence that the time of day of testing 
was related to moan scores on the tests, but on the sixth test there v:as 
some suggestion thal/relationship might exist. 

I'iske raised the question regarding the relationship bet'ween the var- 
iability index and the value of the mean. The correlations between the var- 
iance index and mean index for each of the tests here v/erei A . 4 I, PC *55 > 
KP .50, PS -.26, SC .57, V .05. Five of the coefficients are significant 
three are positive and moderately high; one is negative. Examination of 
two of the bivariate distributions provided no evidence that the variance 
indices were restricted at the low and high ends of the distributions of 
mean indices and the relationships appeared rectilinear. The fact that per- 
sons with high scores tended to be more variable provides some support for 
the hypothesis that variability extends the opportunity for the development 
of adaptive behavior. 

The questionnaires completed by the subjects at the end of the experi- 
ment suggested that they \/ore v/ell rnotiva,ted throughout the experivnent and 
elghty-thrce percent of them reported thet they consistently put forth all 
of their effort in doing as well as they could. 2Iinety-t\.'o percent of uhern 
reported that they were able to v/ork on these tests much more effective '.y 
on some days than they could on others. Eighty-eight percent reported that 
on the whole they enjoyed taking the tests. The test they enjoyed least 
was the Aiming test and the tost they enjoyed next least was the Pumber 
Facility test, the two tint provided the bcDst vari.ance indices. 
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Immediately alter the last form of the last test was administered, 
the students responded to an open ended question asking what they thought 
the real purpose of the experiment v/as. Thirty-three percent provided the 
explanation given at the beginning of the experiment, that v/as, to compare 
technology students to other students. Eighteen percent of the students 
reported that their perception of the purpose of the experiment v/as related 
to the consistency of behavior. Other reported purposes related to describ- 
ing technology students, learning and improvement, motivation, and eye 
movements. Only four students reported that tlioy did not know \^rhat the pur- 
pose of the experiment v/as. At the end of the quosuionnairo , students v/ere 
presented v/ith a check list of five items pertaining to the purpose of the 
experiment. In responding to tViis list, fifty-seven percent of the subjects 
checked the item, "The experiment v/as concerned v/ith the consistency of ny 
test behavior" , thirty- three responded that the puirpose ,v/as , "To determine 
how well I did on these tests in relation to my fellow students also taking 
the tests." These figures su.ggest that a reasonably large proportion of 
the subjects had some realization that the experiment was concerned v/ith 
the consistency of behavior but there v/as nothing to indicate that the sub- 
jects for the most part v/ere strongly motivated to behave consistently, 

A supp3.ementary analysis suggested that the indices of intra-individual 
variability used here are related to the predictability of students’ aca- 
demic behavior. For example, groups divided on the basis of the total 
vai/ianco index derived from the twenty forms of all of the six tesus 
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(120 scored), into hi{jh and low va 3 :iability groiips did differ in predicta- 
bility. The averar;o error of i? 3 ?ad 0 point prediction for the high variabil- 
ity group v;as .28, for the lov/ variability g-roup .009, a. difference sta- 
tistically significant betv/een the .01 and .09 levels. 

Conclii.sions 

Large and significant differences are found aniong individuals in their 
variability of behavior over tihie. For exa;ap].e, one subject had a mean 
score of 49 for Jho tv;enty forsns of the imiTiber Facility test, v/ith a standard 
deviation of 1.75, and another subject v/ith the same mean score had a standard 
deviation of 4.O7. On each test large individual differences are found in 
variability over time. 

The reliability v/ith vMch these differences can be observed varies 
from task to task and the tv/o tasks providing the most consistent varirmce 
index were the Aiming and Number Facility tas-cs. These were the two tasks 
that placed the students under the most stress insofar as they were the tv/o 
least preferred by the subjectsi. 

Variability over time on some tasks is related to variability on other 
tasks, but these relationships are no more than moderate, even taking into 
account the relative inadequacies of the means of observation, and the 
highest correlation between any t\/o of the vs.riance indices was only about 

.50. 

If an easily observable variability chardcteristic hai been identified 
extending over th.e six tests, one would face a difficult problem related to 
the highly speeded natu3?e of the six test^-.. One then would have to dote/:- 
mine the extent to which such, a gonorali'/.cd intra-individual var.i.obi:'.iti' 
was related to variability in speed performance, rather than to vsiriability 
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over different taeks. The relatively small relationship*-; ainon^’ variances 
observed here v'ell rji{.:ht be due to the cona.ion eleirient of speed characber- 
iaias’ all of the tasks and one inir;ht be justified here concluding only 
that to soTie extent tlic speed v;ith ■.■;hich persons perforn tasks shov/s some 
consistent intra-individaal variability, quite apart from the task involved* 
Hov-evor, all of the tasks v/ere speeded and if this in tra -individual var- 
iability were primarily a function of variations in speed, one v;ould expect 
{jre:*ter consistency among variance within tasks. 

The obt^iinod results suggest that at least two of the tasks studied, 
Aiming and Humber Facility, can provide adequate measures of intra-individiAal 
variability. The next question asked earlier by Fiske, is, "V/ith v/hat are 



these individual differences in variability associated?" 
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Intcrcorrelations Betv/een the Six "V(;.riance’‘ Indices (using Rav/ Scores) 
for 100 University of Minnesota Institute of Technology Pre3h!U3n 
Each Taking Twenty Forms of Each of Six RPM Tests. 





FC NF PS 


SC 


V 


Mean 


SP 


Aiming (a) 


-.02 .47-““"- .01 


.28-'<~-- 


.01 


202.04 


167.48 


Flexibility of Closure (FC) 


.17 -‘15 


.16 


.52->:--- 


19.09 


5.45 . 


Number Facility (NF) 


-.18 


.22"- 


.27-»-^- 


26.58 


25.11 


Perceptual Speed (PS) 




.14 


-.01 


58.11 


21.54 


Speed of Closure (SC) 






.06 


58.45 


25. 88 


Vi.;ualization (V) 








41.59 


18.25 



* p <;.05 

♦ 

** P <.01 
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Ta^ble 4 



Intorcorrelations Uetv/cen the Seven "Va'/’iance’' Indices (Using T Score 
for 100 University of Minnesota Institute of Technology Freshmen 
Each Taking Tv/enty Forms of Each of Six KPM Tests 




(Also Included is the "Total Variance" Index) . 



FC W 

Aiming (a) .00 .49‘^"''* 

Flexibility of Closure (FC) .14 
Number Fa.cility (liP) 

Perceptu;il Speed (PS) 

Speed of Closure (SC) 

Visualization (v) 

Total Variance (TV) 



PS 


sc 


V 


TV 


Mean 


su 


•" • Oo 




.05 


.42*>t 


44.89 


36.62 


-.10 


.22-^ 


,27-x-^- 


.15 


40.84 


19.25 


-.10 


.20^ 




.50^-^'^ 


18.23 


18.36 




-.01 


.03 


O 

• 

1 


39.15 


28.02 






.03 


.21* 


54.60 


25.26 








.28*^- 


55.76 


16.97 










64.69 


25.97 



* P C.05 

p <.01 
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Int'^rcorre’’ rns Bo'bvreGn ‘bho Si'-: Vr'.riance Indice^'^ U'^in.^ Bav' Scc?:-o:.: , 
Correlations Corrected for Attenuation (UnroliaBility of Variance Indices) 
Usin.^' Spearman-Brov;n Corrected Reli-?."bilit'/ Coeificicntr. . 

(B=100) 





PC 


FP 


PS 


0 \j 




Ai'.nin;; (a) 


o 

• 

! 


.5:5 


.01 


.35 


.02 


Plexil'ility of Closure (VC) 




.24 


-.23 


.25 


.67 


IteBer facility (iTP) 






-.23 


.28 


.46 


Perceptual Speed (PS) 








.20 


-.02 


Speed of Closure (SC) 










.11 



o 

ERIC 



Intra-individual Temporal Variability 
and Predictability^ 

Ralph F* Berdie 
Office of the Bean of Students 
University of Mnnesota 

Although the assumption that behavior is predicatable does not require 
that behavior be consistent, consistency of behavior may facilitate its 
predictability. If some person’s behaviors are more consistent than those 
of other persons, they may be more predictable. If one person leaves his 
apartment for work every morning exactly at eight o'clock, and another 
person’s time of departure varies widely between seven- thirty and eight- 
thirty, then predictions regarding when the men would return from lunch, 
based on the time they left for lunch, presumedly might be more accurate 
for the former than the latter individual. Persons v/hose habits are regu- 
lar, or v/hose behavior is consistent over time, may be the most predictable. 
An informative review of the research on predictability has been presented 
by Tolbert (1966). Piske and Maddi (I96I) discuss in detail the concepts 
of variability. 

^This study was supported with funds from the U.S.O.B.-O.E.C.-3~7~0^8694”“ 
2082. Appreciation is expressed to Hr. Richard Arvey and Miss Biane 
Johnson for their contribution in analyzing the data-. 
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Only suggestive evidence pertains to the relationships betv/een con- 
sistency and predictability and two studies provide some insight. This 
author (Berdie, 1961 ) reported that the correlations between college apti- 
tude test score and first quarter college grade point average were .70 
for a group of students with little variability shown on responses to a 
mathematics test and .44 for a matched group with high variability. Host 
of the analyses in that study provided results that failed to attain sta- 
tistical significance, but the results were consistent and the conclusion 
was that intra-individual differences in variability were perhaps related 
to . predictability. A later duplication of this study with somev/hat similar 

methods provided no confirmation. 

A more recent unpublished study by Arvey showed that 200 college 
students could be divided into two groups on the basis of the variability 
of their high School grades. The correlation between predicted college 
grade point average based on test scores and high school rank, and obtained 
grade point average for the most variable group was .10, for the least 
variable group .69. Similar correlations based on an independent cross- 
validation sample were .23 and .74. Individuals whose behavior was more 
variable in high school were less predictable in college and intra-indi\ddual 
variability was regarded as a practical moderator variable. (Saunders, 1956) 

The use of a moderator variable in prediction depends on the pur- 
poses of prediction and the situation. Sex often is a moderator variable 
and academic prediction for girls tend to be more accurate than for boys 
(Seashore, 1962 ). A multiple correlation betv^een first year grades and three 
predictors v/as .46 for men and .57 for women in the University of Minnesota 
Arts College. For the total group, R = .52. Although the moderator variable 
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could provide better predictive efficiency for one group, it really did not 
contribute to selection for the total group. For predictive purposes in 
advising and counseling, hov;ever, it did effect probability statements, and 
it did provide incentive for further analysis of the prediction process. 

If the classification of persons on the basis of their consistency of 
performance is to aid in improving predictions for sub-groups of these per- 
sons, then the question of the generalizability of such consistency becomes 
important. To what extent can consistency of behavior be regarded as a 
trait v/hich characterizes the individual? 

Using seven intra-individual variability indices based on six tests, 
the author (Berdie, 19S^) found that the intercorrelations between the in- 
dices ranged from -.18 to .49. Of the fifteen correlations between variance 

indices of the six tests, five attained statistical significance beyond the 

# 

.05 level and the variance indices for five of the tests were significantly 
correlated with the variance index of at least one of the other tests. For 
one test the variance index was not correlated with the index of any of the 
other tests. On these six repeated psychological measurements, the evidence 
suggested that consistency of behavior is not highly specific to each task, 
but also that currently one cannot conclude that a highly generalized consis 
tency of behavior is observable over several tasks. 

Arvey observed that the correlations between variance estimates of 
high school grades in six subject matter areas, including a total variance, 
ranged from .04 to .34. Of the fifteen intercorrelations, including the 
part-whole correlations, nine v/ere statistically significant behond the .05 
level. These results suggested that persons whose grades are variable in 
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high school in one subject tend to obtain variable grades in other subjects 
but the relationships are low and reflect no easily observed generalized 
variability trait. 

Before one can study the usefulness of indices of intra-individual 
variability as moderator variables to improve prediction, the behavior or 
behaviors on which the variability indices are to be based must be selected. 
The present research was concerned first, with the identification of appro- 
priate variability indices, and secondly, with the relationships between 
these indices and predictability of academic behavior. The first concern 
is discussed elsewhere (Berdie, 196?) • The purpose of this report is to 
describe the methods explored in studying relationships between variability 
and predictability and present the evidence revealed. 

Method 

# 

Subjects 

The subjects were Institute of Technology freshmen entering the TJni- 
versity of Minnesota in the Fall of 19^6. All freshmen were told of the 
possibility of participating in the experiment and subjects were selected 
from vol\inteers on the basis of class schedule, proximity to campus, and 
availability of data. The subjects tended to be somewhat superior academ- 
ically to the total entering class because the experiment was conducted 
during the second academic quarter and included only students who survived 
the first quarter. These students v/ere a representative group of bright 
college students motivated to earn forty dollars by participating in an 
experiment that would cause them no stress or discomfort. 
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Tasks Providing a Basis for Inferring Intra-indvidual Variability- 

Subjects were classified in terms of intra-individual variability on 
the basis of their performance on six of the brief, highly speeded, Repeti- 
tive Psychometric Measures developed by Moran and Mefferd (1959)* These 
are relatively pure factorial tasks frequently described in the factor 
analytic literature. 

The Aiming test (a) consists of fifteen rows each containing tv/enty 
circles connected sequentially by a line. The subject places the test 
on a piece of corregated paper and punches holes inside as many circles as 
possible without touching the circle. Subjects used a stylus consisting 
of a pencil sized piece of wood with a thin metal point. The Flexibility 
of Closure test (PC) requires the subject to copy thirty-six geometric 
• figures into matrices of dots. The task, as described by the authors, is 

to retain the image of a specified configuration despite the influence of 
other distracting configurations in the perceptual field. The Number Fa- 
cility test (NF) consists of ninety problems each requiring the addition 
of three two digit numbers. 

The Perceptual Speed test (PS) consists of rows of thirty digits with 
a digit in the left hand column of each row encircled and the task is to 
cross out every digit in the row similar to the encircled digit. The time 
limit specified by Moran and Mefferd for this test is two and one ha3f 
minutes but early experience suggested that too many subjects completed the 
test with this time limit and the time limit was reduce(3 to one and one half 
minutes. 
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The Speed of Closure test (SC) measures the ability to unify an appar- 
ently disparate perceptual field into a single percept* Each form consists 
of tv/enty-tv/o lines and each one has letters in it apparently arranged at 
random but containing from two to four four-letter words which are to be 
encircled* The final test, Visualization (v) consists of tangled lines v/hich 
must be follov/ed visually from their start to their finisn* 

For each of these tests Moran and Mefferd developed twenty different 
forms so that the forms would be e(jui valent* Later research indicated that 
for five of the six tests the alternate forms were reliably different (Moran, 
Kimble, and Mefferd, 19^4) correction factors were provided for the twenty 
alternate forms of the five tests* These correction factors were not used 
in this study in light of the experimental design employed* 

Moran and Mefferd reported reliability coefficients, based on correla- 
tions between form one and form two, for the six tests ranging from *72 to 
*94* For each of the one hundred subjects included in the present research 
a score was derived from the odd numbered forms of each test and from the 
even numbered forms of each test and the correlations between thes- scores 
ranged from *96 to *99* The tests appeared to be reasonably reliable* 

Moran and Mefferd reported that intercorrelations between the six 
tests, using only one form, ranged from *09 to *44* present experi- 

ment a total score was available for each subject on each test, based on 
the twenty forms of the test, and the interco,rrelations betv^een these scores 
ranged from *28 to *65, considerably higher than the intercorrelations re- 
ported by the original authors* Their correlations were based on scores 
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derived from tests requiring from tv^o to five minutes and involved measure- 
ments of considerable less reliability than characterizing the scores used 
here* This raises an interesting question as to the extent to which inter- 
correlations found between tests and resulting factor analytic results de- 
pend on the length and related reliabilities of the tests included in the 
factor analyses. In spite of the higher intercorrelations found here, the 
tests appear to be sufficiently independent from one another to justify 
their use in this research. 

Procedures 

The one hundred subjects were divided into five groups and each group 
was tested at the same time each day for five days during four successive 
weeks* One group was tested at 900 a*®*, another group at 12:50 noon, and 
the three remaining groups tested at later times during the afternoon. Assign- 
ments to time periods were based on students’ class schedules. 

About one fifth of the subjects took form one of the test on the first 
day, form two on the second, etc* Another group of subjects took form five 
on the first day, form six on the second day, and on the tv^entieth day took , 
form four* One group of subjects started on form nine, another group on 
form thirteen, and the final group on form seventeen. Within each tine 
session students vere randomly assigned to sequence groups. 

The experimenter read to each group an introductory statement to 
explain the purposes of the research. Subjects were told the experiment 
was designed to provide comparisons between the psychological characteristics 
as measured by these tests of students in technology and science with 



ch? ;acteristics of other students* The tests were administered by a trained 
and experienced psychometrist who read the test instructions at the first 
session I administered the practice exercises provided by Moran and Mefferdi 
and then administered the tests* Testrng schedules for each group were 
arranged Monday through Friday for four successive weeks and subjects who 
missed sessions made them up during the fifth week* Over ninety-five per- 
cent of the tests were administered to the subjects at the time of day ori- 
ginally scheduled and of the two thousand subject-attendances, 1,924 occurred 
on the day scheduled. 

At the completion of the last form of the last test, each subject com- 
pleted a questionnaire reporting his reaction to the tasks and his percep- 
tions of the purpose of the experiment* In response to an open-ended ques- 
tion, 18 percent of the students reported that they thought that the putpose 
of the experiment was related to the consistency of behavior but later, 
in responding to a check list containing five items pertaining to the re- 
search purpose, percent checked the item, "The experiment was concerned 
with the consistency of my test behavior*" These figures suggested that a 
reasonably large proportion of subjects had some realization that the 
experiment vas concerned with the consistency of behavior but little in- 
dicated that the subjects were strorgly motivated to behave consistently* 

They did report high motivation to perform well* 

The experiment was conducted in a v/ell isolated sub-basement room v/ith 
over-head lights and lamps arranged so that illumination was adequate* 

Little distraction occurred. Subjects were seated in the center of the 
room in classroom chairs with arm tablets* 



- 9 - 



Prediction Data 

In addition to the one hundred and twenty Repetitive Psychological 
Measurement test scores, additional data were available for subjects* 

Those included a score on the Minnesota Mathematics test, a score on the 
Minnesota Scholastic Aptitude test, high school percentile rank, obtained 
Pall quarter grade point average, and predicted Pall quarter grade point 
average derived from the mathematics test* Prom the last two indices a 
discrepancy score was obtained by subtracting the obtained grade point 
average from the predicted grade point average* 

The Minnesota Mathematics test was developed as a means for admitting 
students to the University of Minnesota Institute of Technology and is a 
comprehensive examination covering high school mathematics, with emphasis 

on algebra* Correlations between this test and Pall quarter grade point 

# 

average range about *50 for groups of freshmen admitted to college on the 
basis of information other than this test score* The Pall quarter grade 
point average was based on the grades of all courses taken during the first 
quarter in the Institute of Technology* Students customarily register for 
from three to four courses requiring a total of fifteen to tv^enty hours 
per week of class and laboratory attendance* Por the one hundred subjects 
the correlation between the Minnesota Mathematics test score and the ob- 
tained Pall quarter grade point average was *53 and the results of the ex- 
periment must be interpreted in light of the relatively low predictive 
efficiency of this instrument* 
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For each student the predicted Fall quarter grade point average v/as 
obtained by using a single variable regression equation based on the mathe- 
matics test scores derived from a previous class of Institute of Technology 
freshmen. For the one hundred subjects the correlation between the mathe- 
matics test score and the predicted Fall quarter grade point average was .84. 
The difference between this and a correlation of 1 reflects possible change 
in the population and the extent to which error has been incorporated into 
the original regression equation. The reliability of the obtained Fall 
quarter grade point average can be inferred from the correlation betv/een 
obtained Fall quarter grade point average and obtained Winter quarter grade 
point average I .70. 

Analysis 

The twelve thousand Repeated Psychological Measurement tests were • 
scored by research assistants; scoring was checked; and when necessary 
papers were re-scored. Test scores and other data then were entered on 
basic record cards for each student, verified, and then punched and veri- 
fied on IBM cards. 

For each of the six tests a variability index was computed for each 
student. This consisted of the variance (SD ) of the twenty raw scores 
derived from the twenty forms of the tests. Then, in order to facilitate 
comparisons betv/een tests and to provide a basis for obtaining a total 
variance index, each raw score was transformed to a standard score, using 
a mean equivalent to 5^ and a standard deviation equivalent to 10, based 
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on the distribution of one hundred scores of each form of each test. To 
illustrate this, the mean and standard deviation were calculated for the 
one hundred scores on form one of the Aiming test and for each student, 
the raw score on form one was transformed to a standard score based on this 
distribution. Using the standard scores for each of the six tests variabil- 
ity indices were computed and a seventh variance index was calculated for 
each student, based on all one hundred and twenty standard scores. 

The consistency of the variability indices was analyzed and an odd/even 
reliability coefficient obtained for each of the six tests and for the 
total variance index. The raw scores on each of the ten odd-numbered forms 
were used to obtain one variance index and the scores on each of the ten 
even-numbered forms were used to obtain a second index and the correlations 
obtained for the six tests were: A .85> .4I1 bT *80, PS .55i SC 

The comparable correlation for the variance index encompassing all 120 standard 
scores was .89* 

The variance indices based on five of these tests and on the total 
score v^ere sufficiently reliable to suggest that these measures of varia- 
bility themselves were consistent. 

Table 1 presents the intercorrelations between the six variance indices. 
Six of the fifteen correlations are statistically significant and the var- 
iance on five of the tests is related to the variance on at least one other 
test. The variances based on the six tests and the total variance, which 
provides a part-whole correlation, correlate between -.05 and .49* 

These intercorrelations sviggest that variability over time on some 
tasks is related to variability on other tasks, but these relations liips 
are no more than moderate, even when one considers the reliabilities of the 

observations. Intra-individual variability is not specific to each task 

o 
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and neither is theT'e a strongly generalized characteristic of variability 
that extends over a broad variety of tasks. 

These results suggest that at least two of the tasks studied, Aiming 
and Number Facility, and the total variance index, may provide adequate 
measures of intra-individual variability. 

The most comprehensive variance index, and the one with the highest 
reliability, was the one based on all one hundred and tv/enty standard scores. 
Although this measure of intra-individual variability encompassing six 
different tasks is difficult to interpret because of its heterogeneity, 
it does provide one means for testing the hypothesis that intra-individual 
variability is related to predictability of behavior. 

To test this hypothesis, the mean of each student *s one hundred and 
twenty standard scores and the variance of these scores were recorded and 
the one hundred* subjects were divided into four groups, those with both 
the means and the variances below and above the group average, those v/ith 
the means below and the variances above the group average, and those with 
the means above and the variances below the average. Then groups were 
recombined so that one group included all of those v/ith mean indices above 
the group average, one v/ith mean indices belov/ the group average, one v/ith 
variance indices above the group average, and one with variance indices be- 
low the group average. 

For each of these eight groups statistics were calculated, including: 
the means for the mathematics test, the predicted Fall quarter grade point 
average, the obtained Fall quarter grade point average, the discrepancy 




betv/een these, the correlations between the mathematics test and obtained 
Pall quarter grade point average and predicted grade point average and 
obtained grade point average* 

Then for the total group of one hundred subjects correlations were 
calculated between all of the variables, including the correlations be- 
tv/een the several variance indices and discrepancy betv/een obtained and 
predicted grade point average. 

Then, in light of the demonstrated interactions between the variance 
index and the mean index, the total group of one hundred was divided into 
four sub-groups: students with correctly predicted grade point averages, 

which were high, those with correctly predicted grade point averages which 
were low, those whose grade point averages were under predicted, and those 
whose grade point averages were over predicted. The variance indices on 
the six tests were compared for these groups (Hobert & Dunnette, 196?) • 

Then the group was divided into four other sub-groups consisting of 
subjects with both low mean scores and low variance indices on the Aiming 
test, those with low mean scores and high variance indices, those v/ith high 
mean scores and low variance indices , and those v/ith high mean scores and 
high variance indices, and the grade point discrepancies for these groups 
were compared. 

Then the entire group was divided into two groups, one with above 
average variance indices on the Number Facility test and one with below 
average variance indices on this test, and the groups were compared on the 
basis of correlations and means. 
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In ord.Gr ijo allov/ for possiblG GfiGct of sIzg of raw scorGS on thG 
tGsts, anothGr group of studonts vjas SGlGctod consisting of thosG v;ho had 
moan scorGS on both thG Aiming and Numbsr Facility tests that placed them 
within plus and minus one standard deviation of the means for the total 
groups and for this group correlations were determined between variance 
indices and other characteristics, including grade point discrepancies. 

Next, for each student the coefficient of variation was calculated 
on the Number Facility test and comparisons v;ere made of students with 
coefficients of variation above the median and those belov: the median. 

These various analyses were designe to provide information concerning 
the differences in predictability of stu ents with different variability 
characteristics . 

Results 

# 

Consideration of Variance and Mean Indices 

Table 2 shows the characteristics of the one hundred subjects divided 
into eight sub-groups on the basis of individual subject mean and variance 
on total score derived from responses to all twenty iorms of each of the 
six tests. The first group of tv;enty-seven subjects had a mean standard 
score for the one hundred and i»wenty tests and a variance of these scores 
placing them at or above the group mean average and group mean variance 
for the total group of one hundred subjects. The second group had mean 
scores above the group average but variance indices below the group aver- 
age and the next two groups consist of thirty students with both means and 
variances below the group average and twenty-two v/ith means below and var- 
iances above the group average. The fifth, the high mean group, consists 



of the first and second groups comhined, the next group, the low mean group, 
of the third and fourth groups combined, the next ’’high variance” group of 
the first and fourth groups combined, and the last ’’low variance” group of 
the second and third groups combined. 

The hypothesis, ’’Subjects v/ho were more variable are less predict- 
able” can be tested by looking at the correlations between predictors and 
criteria and also by looking at the discrepancies between predicted and 
obtained grade point averages, Por the low variability group, the eighth 
group, the correlation betv/een mathematics test score and the obtained 
Pall quarter grade point average was ,39i as compared to the correlation 
of ,28 for the high variability group, group seven. The difference is in 
the expected direction but is not statistically significant. 

The mean discrepancy between predicted and obtained g'^ade point average 
for the low variability group was ,01 with a standard deviation of ,65i as 
compared to the mean discrepancy of ,28, with a standard deviation of ,6l, 
for the high variabili;ty group. The ”t” for the mean difference was 2,19i 
showing that the difference was statistically significant between the pro- 
bability levels of ,05 and ,01, The discrepancy analysis supports the 
hypothesis that persons who have low variability indices will perform more 
as predicted compared with persons with high variability. 

The two variability groups were quite similar on the basis of mean 
mathematics test scores and predicted grade point average. The high var- 
iability group obtained somewhat better grades on the average than did the 
low variability group but the difference v/as not statistically significant. 
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The correlational comparisons of the four groups divided on the basis 
of both variance and means reveal for the groups with above average mean 
indices, that the low variability group is more predictable than the high 
variability group. The correlations between the mathematics test and the 
Pall quarter obtained grade point average for the former group was ,67i 
for the latter group .16* Transformation of these coefficients to and 
evaluation of the significance of the difference provide a ”t" of 2.08, 
significant with a probability between .05 and ,01. When the mean discre- 
pancies between predicted and obtained grade point averages are compared, 
.14 and .32, the differences are not statistically significant. 

When the low and high variability groups with below average mean 
indices are compared, the correlations are almost identical, .55 and .5^, 
and the discrepancies .01 and .23 i are not statistically significantly 
different. Thus, the hypothesis has some support from this analysis but 
the results suggest that the hypothesized relationship between variability 
and predictability is observable mainly for subjects achieving high mean 
scores on the tasks from which the variability index is derived. 
Intercorrelations of all Variables 

Table 1 shows the means, standard deviations, and intercorrelations 
for all the variables, with the RPM indices based on standard scores, A 
similar table using raw scores revealed essentially the same relationships. 
Obtained Pall quarter grade point average, the criterion used here, was 
substantially correlated with discrepancy between obtained and predicted 
grade point average and significantly correlated v/ith mean scores on the 



Aiming, Numljer Facility, Perceptual Speed, Visualization, and total mean 
score on the EPM tests. Obtained Pall quarter grade point average was not 
correlated significantly with any of the variance indices. 

The GPA discrepancy v/as correlated .19i with the variance index on the 
Visualization test, but this was not statistically significant. The dis- 
crepancy was correlated with the mean score on three of the six tests and 
with the total mean score. The absence of significant correlation between 
the discrepancy and the seven variance indices does not lend support to the 
geneivil hypothesis and in light of the results presented in Table 2, again 
suggests that the relationships, if present, are quite complex. 

The mean indices are significantly correlated with the variance indices 
on five of the six tests and this correlation for the Perceptual Speed test 

is moderate and negative. For the total index the correlation between var^ 

# 

iance and mean is .21, barely significant at the .05 level. 

Consideration of ^Relationship Between Predictor and Criterion 

The relationship betv/esn predicted and obtained grade point average 
and the seven mean and seven variance indices is analyzed in another way 
in Table 5« A scatter diagram v/as prepared showing the relationship be- 
tween predicted and obtained grade point averages and the group mean pre- 
dicted and group mean obtained averages were determined. The total group 
was divided into four sub-groups corresponding to the four quadrants. One 
quadrant contained thirty-one subjects v/ith both predicted and obtained 
high grade point averages, another contained thirty-six subjects with both 
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predicted and obtained low point averages. One quadrant contained 

eighteen subjects whose obtained predicted grade point average was high 
and predicted grade point average low, and the forth quadrant contained 
the fifteen subjects with high predicted grade point averages and low ob- 
tained grade point averages. For each of these groups, Table 5 presents 
the iP.ean and standard deviation for each of the seven variance indices and 
for each of the seven mean indices. The subjects with the most accurately- 
made predictions are in the first two groups , the ones with the least accu- 
rate predictions in the last two groups, Examina.t.ipn of the average var- 
iance indices shows few consistent or meaningful trends. When one combines 
the sixty-seven subjects in the two grr aps least accurately predicted, the 
correct prediction group has the highest mean variance on the Aiming, 
Flexibility of Closure, and Number Facility tests and the incorrect pre- 
diction group lias the highest mean variance on the Perceptual Speed, Speed 
of Closure, and Visualization tests. The mean variances for the two 
groups on the total measure are practically identical. 

On the first three tests the mean variances for the low achievers 
who were correctly predicted are higher than those of the other three 
groups, but this is not found for the last three tests, V/hen the two 
groups with low predicted grade point averages are compared, the group 
that achieved according to prediction and tLe group that did better than 
predicted, a difference in the mean variance on the Aiming test is statis- 
tically significant with a probrbility of less than ,01 and the variances 
also are significantly different for the variance index on this test but 
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this does not appear to fit in v;ith any of the other results. Perhaps if 
any differences can be observed in this table, they refer more to differ- 
ences between high and low achievers than to differences between 'correct 
and incorrect predictions. 

Consideration of Mean and Variance Indices on Aiming 

Table 4 presents the characteristics of four groups divided on the 
basis of variance index and mean index on the Aiming test, using raw 
scores. Thirty subjects had both mean and variance indices below the group 
mean; eighteen had mean indices '^elow and variance indices above the group 
means, etc. When the two groups of students with low mean indices are com- 
pared, those with low variance and those with high variance indices, the 
difference in the mean discrepancies is statistically significant beyond 
the .05 level with the high variance group having the greatest discrepancy. 
When the two grWps with high mean indices are compared, again the high 
variance group has the greatest discrepancy, but the difference is not 
statistically significant. In Table 2, the statistically significant* dif- 
ference between the high and low variability groups was found for the groups 
with mean indices above the group average; with the Aiming test, the signi- 
ficant difference was found for the low mean index group. Again some, but 
far from conclusive evidence is available concerning the relationship be- 
tv/een variance and predictability. 

Variability on Humber Facility 

Table 5 shows characteristics of the subjects divided on the basis of 

variance index on the Humber Facility test. Of the ninety- three subjects 

« 
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foi: v/hom complete data v/ere available for this analysis, forty-seven v/ere 
in the high variability group and forty-six in the low variability group* 

For these two groups correlations betv/een the predictor test and obtained 
grade point average were almost identical* The discrepancy betv/een ob- 
tained and predicted grade point average was slightly greater for the 
variable group than for the less variable group, but the difference was not 
significant. 

Middle Range Subjects 

Table 6 shov/s the characteristics of students who on both the Aiming 
and Humber Facility tests had mean scores that placed them within plus and 
minus one standard deviation for the total group mean. Again no significant 
correlation appeared betv/een the predicted and obtained grade point average 
and the variance indices of these tv/o tests. When the twenty-eight subjects 
with lowest variances on the Aiming test were compared with the twenty- 
eight subjects with highest variance, the low variability group had the 
highest correlations between predicted and obtained grade point average, 
a correlation of *58 as compared to a correlation of *40 for the other group* 
When the groups were divided on the basis of variability on the Number Fa- 
cility test, again the lov/ variance group had the highest correlation, #58, 
as compared to the correlation of *46 for the other group. V/ith groups as 
small as these, differences between correlations to be significant would 
have to be much larger than one could reasonably expect to obtain so the 
results are only mildly suggestive* 

Relative Variability 

Finally, in order to study the possrble effect of the relationships 
between the mean and variance indices themselves, the use of the coefficient 
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of variation vias explored and the results are presented in Table 7* 

The ninety- three students for whom complete data v/ere available were 
divided on the basis of high and low coefficients of variation on the 
Number Facility test. For the variable group the correlation betv/een 
predicted and obtained grade point average was .26, for the less variable 
group .55* The correlation between the Mnnesota Scholastic Aptitude 
test and grade point average for the variable group was .10, for the less 
variable group .55* The correlation between the Minnesota Scholastic Apti- 
tude test and high school percentile rank, also differed although not sig- 
nificantly. The only correlations that differed significantly were those 
betv/een the Mathematics test and high school percentile rank, where the 
correlation was -.05 for the variable group, +.34 f’or the less variable 
group. Using this variability index the two groups also differed on the 
basis of mean Fall quarter grade point average, with the less variable 
group achieving better grades. 



Discussion 

Answering the question, ”Is intra-individual variability related to 

predictability?" is not easy. The definition and measurement of intra- 

% 

individual variability are complex. The answer to the question depends 
in part on the tasks used and the derived indices. The function being 
predicted makes a difference, and more than one method is available for 
observing the accuracy of prediction. On five of the six tasks used here, 
the subject’s level of performance was significantly related to his variance 
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and one of these coefficients was negative. The answer to the question 
also depends in part on the segment of the range being observed of the 
performance level of the task providing the variance index. Some of the 
evidence suggests that the answer depends in part on the level being con- 
sidered of both the predictor variable and the predicted variable. The 
results do suggest that predictability* of behavior, as defined here, is 
influenced to some extent by the subject *s intra-individual variability 
but that the relatior.&hip is not a straight-forward one. 

The total variance index based on one hundred and twenty scores is 
most difficult to interpret insofar as it consists of twenty scores on 
each of six relatively independent tests and these are all transformed 
to standard scores. Using this complex index of intra-individual varia- 
bility, the correlation for the less variable group between the predictor 
and the criterion is higher, but not statistically significantly so, than 
the correlation for the variable group. VJhen the mean discrepancies be- 
tween predicted and obtained grade point averages are compared for the 
high and low variability groups, hov/ever, the difference is statistically 
significant. In ligVit of this statistically significant difference and in 
light of the difference in the correlations being in the expected direction, 
the evidence does suggest that this complex variance index is related 
somevzhat to predictability. 

V/hereas the correlation between the predictor and predicted variable 
appears to be relatively easy to interpret, the GPA discrepancy measure 
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is nou. Two methods are available for handling the discrepancy: it can 

be treated algebraically or arithmetically. Treating it algebraically 
pays attention not only to the size of the discrepancy but also to its 
direction whereas treating it arithmetically reflects only the size of 
the discrepancy and ignore its direction. 

The anlysis in Table 2 was done using both the algebraic and the 
absolute discrepancies. V^ereas the difference between the tv/o groups on 
the algebraic discrepancy was significant, the absolute discrepancies were 
the same for the groups, V/hen the two groups with the high means were 
considered, the one with the high variance and the other v/ith the low 
variance, the algebraic discrepancy v/as higher for the high variance 
group. For the two groups v/ith low mean indices, the low variance group 
had the lovrest algebraic discrepancy but the highest absolute discrepancy. 
If relationships between the algebraic discrepancy and the other var- 
iables v/ere curvilinear a more precise picture of the relationship might 
be provided by using the absolute or arithmetic discrepancy. These re- 
lationships were examined for curvilinearity, and no evidence vjas observed 
suggesting other than rectilinear relationships c At the same time these 
observations were made, the relationships between mean and variance indices 
were observed and no evidence v/as found that these were curvilinear. 

The difference in correlations shoi-m in Table 2 for the two groups 
with high means, as compared to the corresponding difference for the two 
groups with low means, is difficult to explain. For the tv/o groups v;ith 
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high means, the group vdth high variance had a standard deviation of 8.20 
on the Mathematics test and *52 on the criterion Grade Point Average. 

The low variance group had a much smaller standard deviation on the Math- 
ematics test, 4 * 72 »and about the same standard deviation on the GPA, .54* 
The interaction betv/een the mean value and the variance value apparently 
has to be considered. 

If the correlations between GPA discrepancy and the seven variance 
indices in Table 1 had proved to be statistically significant, the most 
easily interpreted evidence concerning the hypothesis would be available. 
Obviously, the relationships were not that simple. 

The attempt to consider the interaction between the mean index and 
the variance index by using the coefficient of variation provided results 
that indicated, although often not significantly, that taking into account 
both of the measures, the less variable group was more predictable. The 
coefficient of variability is not a rigorous statistic but v/ithin the de- 
fined limits of this analysis, the results did suggest that the hypothesis 
might v7ell be best tested taking into account measures corresponding both 
to the mean and variance indices. 

Conclusion 

The evidence suggests a relationship between the variability over time 
of a personas behavior on simple tasks and the effectiveness with which his 
academic performance can be predicted. The relationship between intra- 
individual variability and predictability is complex and the measurement of 
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this relationship must consider the subject’s level of performance on the 
tasks from v/hich the variability indices are derived, the level of perfor- 
mance on the predictor and predicted variable, and the method used in de- 
termining accuracy of prediction# 

If intra-individ\ial variability is an effective moderator variable, 
accuracy of prediction perhaps can be increased for the least variable 
group, but prediction for the variable group vail remain inefficient# If 
the efficiency of prediction is to be increased for the total group, and 
particularly for the group v/ith greater intra-individual variability, then 
nev 7 prediction models may have to be devised and nev/ predictors or nevi 
criteria studied# 

For example, for the group of persons vd.th high variability, highly 
speeded tests may be inappropriate as predictors# For this group a recti- 
linear regression model may be inappropriate and the nature of the shape of 
the relationship between predictors and criterion may be different# For 
the least variable group, a criterion based on one quarter of academic per- 
formance may be appropriate whereas for the group with greater temporal 
variability, a criterion of academic success based on several semesters of 
work may be more appropriate# 

Temporal intra-individual variability may prove to be a useful and 
meaningful concept but before this can be determined further , more effi- 
cient means vdll be needed for observing such variability# The results 
here can be interpreted only as being suggestive# They are promising# 
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Intercorrelations Between Variables Using RPM Standard Scores 
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Table 5 

Correlations Botv/een Predictors and Fall Quarter Grade Point Average for 
Groups with High and Low Variances on the RPM Number Facility Test. 



,(1) 




* 


Variance 

above 

median 

47 


Variance 

below 

median 

46 


Correlation between 


Math 


test and Pall quarter GPA 


.34 


.32 


Correlation between 


ESR and Pall quarter GPA 


.46 


.48 


Correlation between 


I/ISAT 


and Pall quarter GPA 


.21 


.15 


Correlation betv/een 


MSAT 


and HSR 


.33 


.26 


Correlation betv/een 


Math 


test and HSR 


.26 


.07 


Math test 




mean 


51.62 


33.07 






S 3) 


5.88 


6.53 


HSR 




mean 


85.91 


89.41 






S D 


10.80 


7.59 


MSAT 




mean 


54.43 


55.15 






S B 


11.09 


12.02 


GPA 




mean 


2.41 


2.56 






S D 


.67 


.70 


Discrepancy between pre- 
dicted and obtained GPA 


mean 


.23 


.14 






S D 


.64 


.63 




and MSA available for only 95 of 100 subjects. 
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Table 6 

Relationships Betv/een Variance and Predictability for ^6 Subjects 

in Middle Ability Range 

f 









Correlation between discrepancy between predicted and obtained 
Pall quarter GPA and variance on Aiming test 

r = -.02 

Correlation betv/een discrepancy between predicted and obtained 
Pall quarter GPA and variance on Number Pacility test 

r = .02 



Correlation between above two variances 



r = -.05 



Correlation between predicted and obtained Pall quarter GPA 

r = .51 

Correlation betv/een predicted and obtained GPA for 28 subjects 
with lowest variance on Aiming test 

r - .58 



Correlation between predicted and obtained GPA for 28 subjects 
with highest variance on Aiming test 

r = .40 

Correlation between predicted and obtained GPA for 28 subjects 
with lov/est variance on Number Pacility test 

r = .58 



Correlation betv/een predicted and obtained GPA for 28 subjects 
with highest variance on Number Pacility test 

r = .46 




These subjects had mean scores on both Aiming and Number 
Pacility tests that placed them v/ithin - S.D. of the means 
for total group. 
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Table 7 

Comparisons of Sutioeots Divided into Two Groups, Those Above and Those Belovf the 
Median Coefficient of Variation on the Humber Facility Test. 

(H=95^) 

Lov 7 coefficient of High coefficient of 
variation group variation group 



• 


N=4^ 

Mean S D 


B=47 

Mean S D 


Minnesota Math test 


33.65 


7.88 


31.04 


6.55 


Predicted Fall quarter GPA 


2.26 


.57 


2.14 


.34 


Obtained Fall quarter GPA 


2.49 


.53 


2.23 


.80 


Difference between predicted and obtained 


. .23 


.45 


.15 


.78 


High school rank 


88.89 


8.88 


86.43 


9.94 


MSAT 


54.17 


10.55 * 


55.38 


12.45 


Correlated variables 










Minn. Math test and Fall quarter GPA 




.46 




.20 


Predicted and obtained Pall quarter GPA 




.55 




.26 


High school rank and Fall quarter GPA 




.48 




.41 


MSAT and Pall quarter GPA 




.35 




.10 


MI-TT and Winter quarter GPA 




.34 




.19 


High school rank and V/inter GPA 




.26 




.52 


MSAT and V/inter quarter GPA 




.15 




.15 


Pall quarter and V/inter quarter GPA 




.65 




.73 


MSAT and ilSR 




.42 




.22 


Vm and HSR 




.34*^ 




-.05 


^High School Rank and 1©AT score available for only 95 subjects. 



*-x- p <.05 




